{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aea52082-935a-49ba-acd1-d6df353f3400",
   "metadata": {},
   "outputs": [],
   "source": [
    "正向传播"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9c01fa11-5b70-457a-bfd2-f28fa6d48e38",
   "metadata": {},
   "outputs": [],
   "source": [
    "正向传播是指神经网络沿着输入层到输出层的顺序，依次计算并存储模型的中间变量（包括输出）\n",
    "对于每一层进行加权求和，同时使用激活函数添加非线性特征，通过激活函数得到第一层的输出H1\n",
    "如果为多层时，则按照上面方法，上一层的输出为这一层的输入"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d19c6d13-8f6e-405d-b149-9f0f32c5145f",
   "metadata": {},
   "outputs": [],
   "source": [
    "反向传播"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1b5104ec-ba33-446c-bc78-94fc7ca01e9f",
   "metadata": {},
   "outputs": [],
   "source": [
    "反向传播是指计算神经网络参数梯度的方法，根据微积分中的链式法则，从输入端到输出端的顺序\n",
    "依次计算并存储目标函数相关的神经网络各层的中间变量以及参数梯度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "82aa20be-cfc6-493f-a584-12411a8756ee",
   "metadata": {},
   "outputs": [],
   "source": [
    "训练深度学习模型\n",
    "正向传播以来当前模型的参数值，而这些模型是在反向传播的梯度计算后通过优化算法进行迭代的\n",
    "另一方面，反向传播的梯度计算依赖于当前的变量值，变量正室通过正向传播计算得到的\n",
    "由于我们反向传播中需要使用正向传播的中间变量的内存因此，正向传播结束后不能立即释放中间内存\n",
    "导致训练内存要大于预测内存。中间变量的数量与神经网络数量相关，与批量相关"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:pytorch]",
   "language": "python",
   "name": "deep_learning"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
